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ABSTRACT 



The National Household Education Survey (NHES ) is a data 
collection system of the National Center for Education Statistics (NCES) 
designed to address a wide range of education-related issues. It is a 
telephone survey of the noninstitutionalized civilian population. In 1995, 
the NHES included a survey of adult education (AE) . This report examines 
errors arising from the interviewing process in the *AE components of the 
NHES: 95. The estimates from this component, and from every survey, are 
subject to both sampling error and nonsampling error. In this analysis, 
measurement errors are estimated by reinterviewing a sample of respondents 
and asking them a subset of the same questions included in the original 
interview. Statistic produced by comparing the original and reinterviews 
estimate the consistency of reporting. In the NHES: 95 reinterviews, 
discrepancies between some of the original interview questions and the 
reinterview questions were reconciled. Reinterviews were completed with 1,109 
cases. For most survey questions, the reporting was consistent, and only a 
few modifications need to be considered for future surveys. However, for 
questions associated with barriers to AE participation, more problems were 
identified. The greatest discrepancies were for questions about work-related 
activities and personal development courses. The reconciliation of the 
responses to the original interview and reinterview did not give much 
information about response bias for the questions, but the results did 
generally support the use of the reinterview as an independent replication of 
the original interview. (Contains 9 tables, 1 exhibit, and 10 references.) 
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1 . 



Background 



The National Household Education Survey (NHES) is a data collection system of the 
National Center for Education Statistics (NCES), which has as its legislative mission the collection and 
publication of data on the condition of education in the Nation. The NHES is specifically designed to 
support this mission by providing information on educational issues best addressed by contacting 
households rather than schools or other educational institutions. The NHES provides descriptive data on 
the educational activities of the U.S. population and offers policymakers, researchers, and educators a 
variety of statistics on the condition of education in the U.S. 

The NHES is a telephone survey of the noninstitutionalized civilian population of the U.S. 
Households are selected for the survey using random digit dialing (RDD) methods and data are collected 
using computer-assisted telephone interviewing (CATI) procedures. Approximately 60,000 households are 
screened for each administration, and individuals within households who meet predetermined criteria are 
sampled for more detailed or extended interviews. The data are weighted to permit estimates of the entire 
population. The NHES survey for a given year typically consists of a screener, which collects household 
composition and demographic data, and extended interviews on two substantive components addressing 
education-related topics. In order to assess reliability and inform future NHES surveys, each 
administration also includes a subsample of respondents for a reinterview. 

The primary purpose of the NHES is to conduct repeated measurements of the same 
phenomena at different points in time. Throughout its history, the NHES has repeatedly collected data in 
ways that permit estimates to be tracked across time. This includes repeating topical components on a 
rotating basis in order to provide comparative data across survey years. In addition, each administration of 
the NHES has benefited from experiences with previous cycles, resulting in enhancements to the survey 
procedures and content. Thus, while the survey affords the opportunity for tracking phenomena across 
time, it is also dynamic in addressing new issues and including conceptual and methodological refinements. 

A new design feature of the NHES program implemented in the NHES:96 is the collection 
of demographic and educational information on members of all screened households, rather than just those 
households potentially eligible for a topical component. In addition, this expanded screening feature 
includes a brief set of questions on an issue of interest to education program administrators or 
policymakers. The total screener sample size is sufficient to produce state estimates of household 
characteristics for the NHES: 96. 



Full-scale implementations of the NHES have been conducted in 1991, 1993, 1995, and 
1996. Topics addressed by the NHES:91 were early childhood education and adult education. The 
NHES: 93 collected information about school readiness and school safety and discipline. The 1991 
components were repeated for the NHES:95, addressing early childhood program participation and adult 
education. Both components underwent substantial redesign to incorporate new issues and develop new 
measurement approaches. In the NHES:96, the topical components are parent/family involvement in 
education and civic involvement. In addition, the NHES: 96 expanded screening feature includes a set of 
questions on public library use. 

In addition to its topical components, the NHES system has also included a number of 
methodological investigations. These have resulted in technical reports and working papers covering 
diverse topics such as telephone undercoverage bias, proxy reporting, and sampling methods. This series 
of technical reports and working papers provides valuable information on ways of improving the NHES. 



1.1 Purpose and Overview of Report 

This report examines errors arising from the interviewing process in the Adult Education 
(AE) component of the NHES:95. The estimates from this component and every survey are subject to both 
sampling error and nonsampling error. Sampling errors, the differences between the population values and 
the sample estimates that arise because data are obtained from only a sample of the population, are 
generally well understood and can be estimated from the survey data themselves. Nonsampling errors, on 
the other hand, arise from a variety of sources and are more difficult to measure. Important components of 
nonsampling error for the NHES:95 include coverage, nonresponse, and measurement errors. 

In this analysis, measurement errors are estimated by reinterviewing a sample of 
respondents and asking them a subset of the same questions included in the original interview. The 
reinterview procedure does not account for all the measurement errors in the interviewing process. For 
example, systematic errors that would be made in both the original interview and the reinterview are not 
discovered with this approach. Rather, the statistics produced by comparing the original and reinterview 
responses estimate the consistency of reporting, assuming both interviews were conducted under the same 
general conditions. A general review of the design and analysis of reinterviews is given by Foreman and 
Schreiner (1991). Brick, et al. (1994) discuss the use of reinterviews in the context of other nonsampling 



When the same respondents are asked the same questions on different occasions, different 
responses may be obtained. Not all the differences are necessarily the result of measurement error. 
Discrepancies in responses can be grouped into four categories: 

• Circumstances related to the topic under study may have changed between the first 
report and the second; both answers, although different, may be correct. 

• The original response may have been recorded (interviewer error) or reported 
(respondent error) incorrectly. 

• The reinterview response may have been recorded or reported incorrectly. 

• Both the original and reinterview responses may have been recorded or reported 
incorrectly. 

In the NHES:95 reinterviews, discrepancies between some of the original interview 
questions and the reinterview questions were reconciled. This means that when the reinterview response 
was different from the original response, the interviewer asked the respondent to verify which response was 
correct. This process of reconciling the responses was done after the reinterview was completed and it was 
done only for selected questions. The purpose of the reconciliation is described in Section 2.2 of this 
report. For questions that were reconciled, there are three different responses available for analysis: the 
original response, the reinterview response, and the reconciled response when the original and reinterview 
responses were not identical. This report uses all of these responses in various parts of the analysis. 



The primary objectives for the NHES:95 reinterview program were: 

• To identify survey questions that were not reliable, i.e., the two interviews did not 
elicit the same response; 

• To quantify the magnitude of the response variance for groups of questions 
collected from the same respondent at two different times; and 

• To provide feedback to improve the design of questions for future surveys. 

An objective in many reinterview programs is to provide a check on interviewers who 
might be recording entire interviews without speaking to the respondents. Since the NHES:95 was a 
computer-assisted telephone interview (CATI) survey operated in a centralized location, there was no need 
to design reinterviews to verify that interviews were genuine. The CATI interviews were routinely 
monitored throughout data collection, and it was highly unlikely that a telephone interviewer could invent 
whole interviews. 



A subset of the original AE questions was included in the reinterview program for the 
NHES:95. This was done to reduce the burden on respondents who had already completed one or more full 
interviews and to prevent asking some questions that were very time dependent. The appendix contains a 
copy of the reinterview questions. In general, they were selected based on the following criteria: 

• Questions that were key statistics or used for calculating critical estimates; 

• Questions required for critical skip patterns or that provided information for 
displays for the subsequent sections and questions; 

• Questions that were not time dependent (for example, it would be inappropriate to 
ask, "During the past week, did you work at a job for pay or income?"); and 

• Questions that were new to the NHES and had not been tested in other surveys. 

Questions were selected from specific subject areas. Those subject areas chosen for the 
reinterview are given below: 

Educational attainment 
High school diploma 

High school requirement through a GED test 
Work experiences in the past 12 months 
Language background 
Participation in adult education 
Adult education programs and courses 
Employer awareness 
Employer support for adult education 
Barriers to participation. 

Section 2 of this report summarizes some of the critical features of the sample design of 
the NHES:95 and the design for the reinterview program. In Section 3, the analysis methods used to 
assess the reliability of reporting reinterview data are described. The gross and net difference rates for the 
NHES:95 reinterview data are presented in Section 4 as well as a discussion of the implications of the 
results for the analysis of estimates from the AE component of the NHES: 95 and the planning for future 
AE studies. The final section summarizes the findings and provides some recommendations for future 
work. 



2 . 



Design Considerations 



2.1 Sample Design of NHES:95 

The NHES:95 was a random digit dial (RDD) telephone survey conducted with persons in 
a sample of telephone households in the 50 States and the District of Columbia between January and April 
1995 using computer-assisted interviewing. First, a screening interview was administered to identify 
households and eligible persons within the households. The study included two components: an Early 
Childhood Program Participation (ECPP) interview of the parents of children from birth to third grade 
(with a maximum age of 10 years); and an AE interview of adults 16 and older and not enrolled in 
elementary or secondary education. 1 The respondent for the ECPP interview was the parent or guardian 
who knew the most about the child's care and education. The respondent for the AE interview was the 
sampled adult. 



For the AE component of the NHES:95, the target population was the noninstitutionalized 
civilian population of persons age 16 and older in the United States who were not currently enrolled in 
secondary school. Since only persons in telephone households were surveyed, the estimates were adjusted 
so that the totals were consistent with the total number of persons in both telephone and nontelephone 
households. Screening interviews were completed with 45,465 households. The estimated response rate 
for the screening of households was 73.3 percent, where the response rate is the percentage of all possible 
interviews that were completed, weighted using the probabilities of selection. Adults were sampled for the 
AE interview in a subsample of these households. Table 1 shows the number of interviews sampled and 
completed, as well as the weighted completion rate (the percentage of sampled eligible adults who 
completed the adult education interview) and the overall response rate (the screener response rate 
multiplied by the interview completion rate). 



In addition to the regular AE interview, a splice sample AE interview was administered in approximately 4,000 
households using the NHES:91 participation questions. The interviews in the splice sample were excluded 
from the reinterview. 



Table 1 . Adult education interviews weighted completion and response rates 





Number 

sampled 


Number 

completed 


Completion 

rate 


Response 

rate 


Households for screening 


59,713 


45,465 


73.3 


73.3 


Adult education 


24,538 


19,722 


80.0 


58.6 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey 
1995. 



2.2 Reinterview Design 

Reinterviews with the respondents to the original interview were designed to supply data 
about the reliability of the data collected. A random sample of completed AE respondents was selected to 
accomplish this objective. However, not all interviews were eligible for selection for a reinterview. 

Households that had been sampled for other special methodological samples were excluded 
from the reinterview sample. These special samples included the test of the expanded screener, the splice 
sample AE interview, and the Bias Study. The expanded screener was a test of the feasibility of collecting 
demographic monitoring information and a brief substantive component from all households. The splice 
sample AE interview administered the NHES:91 participation questions instead of the NHES:95 
participation questions to examine whether the different designs of the AE screening procedures and 
questionnaires influenced estimated participation rates. The Bias Study examined the potential size of the 
bias of NHES:95 estimates using an intensive reinterview procedure. The goals of the Bias Study are 
closely related to the reinterview study, but the methods used to obtain the data were very different. The 
Bias Study is described in detail in Brick, et al. (1996) and will be referenced as the Bias Study in the 
remainder of this report. 

One last exclusion for the reinterview sampling involved language problem cases. If the 
original interview was coded as a language problem case but later completed (perhaps by a bilingual 
interviewer), the case was excluded from reinterview sampling. This restriction was at the household level 
rather than at the individual interview level; i.e., if any interview in the household was a language problem 
case, then the household was not eligible for reinterview sampling. 



The remaining households and the AE interviews within these household were eligible for 
sampling for reinterview once all of the interviews in the household were completed (all completes or 
ineligibles). Thus, if some of the interviews in the household were not completed and others were 
completed, none of the completed interviews in the household were eligible for reinterview sampling. This 
occurred most often when the sampled person could not be contacted at a convenient time to complete the 
interview. This restriction in the sample was implemented to prevent the reinterview activity from 
disrupting the completion of the original interviews. 

Table 2 details the number and percent of households eligible for reinterview sampling as 
well as the reasons for ineligibility. The first part of the table shows that 12 percent of the sampled 
households were excluded because they were sampled for the splice sample or the expanded interview 
sample. This random sampling was done prior to collecting any data from the households so the exclusions 
are from all sample households. The other exclusions given in the bottom portion of Table 2 were 
implemented by sampling from persons who had completed the AE interview. The AE interviews that were 
excluded at this stage would have been eligible for sampling if other conditions were satisfied (e.g., if all 
the other interviews in the household had been completed). As a result of this type of nonresponse, bias 
could be introduced into the estimates. For example, cases that were finalized but were not complete or 
ineligible (e.g., final refusals) could have different characteristics than the completed cases. In all, 29 
percent of the households that had one or more adults sampled for AE were excluded from the reinterview 
due to these reasons. 

Once the household and the interviews within household became eligible for reinterview, 
specific adults were sampled. The sampling for reinterview was not done until at least 2 weeks after all 
original interviews in the household were completed so that respondents were unlikely to simply remember 
and repeat their earlier responses. Foreman and Schreiner (1991) discuss the issue of the time between the 
original interview and reinterview. No analysis of the effect of the time difference was undertaken in this 
study. This time restriction was relaxed at the end of the data collection period so that all eligible AE 
interviews could have an opportunity to be sampled. 

The only other step in sampling was designed to limit the response burden on a household. 
No more than one case was sampled for reinterview from the same household, although any of the 
completed AE interviews in the household could have been sampled for the reinterview. Since 98 percent 
of the households sampled for AE had only one adult sampled for AE, this limitation was very minor. 



Table 2. Households eligible for reinterview sampling, NHES, 1995 



Total completed screeners 

Splice sample - ineligible 
Expanded screener - ineligible 
Total ineligible 

Remaining eligible 


Number of 
households 


Percent of households 
with completed screeners 


45,465 

4,082 

1,478 

5,560 

39,676 


100.0% 

9.0% 

3.3% 

12.2% 

87.8% 






Percent of households 




Number of 


with at least 1 adult 




households 


sampled 


Households with at least one adult sampled 


23,275 


100.0% 


Exclusions from sampling: 






Finalized, but not complete or eligible 


3,618 


15.5% 


Language problem 


2,173 


9.3% 


Not finalized 


805 


3.5% 


Sampled for Bias Study 


229 


1.0% 


Total eligibles excluded 


6,825 


29.0% 


Total eligible for reinterview sampling 


16,450 


71.0% 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey 
1995. 



The interviews were sampled for the reinterview at different rates for AE participants and 
nonparticipants. The rates were set to achieve sample size goals: for the participants, the goal was to 
complete 750 interviews; for the nonparticipants, 250 interviews. These goals were set to obtain enough 
reinterview cases to be able to analyze questions that were only asked for certain types of participants (e.g., 
those in work-related activities) and for questions asked only for nonparticipants. The overall samp lin g 
rate for participants was 1 0 percent and the rate for nonparticipants was 4 percent. 



A sample of 1,289 cases was selected for reinterview. A total of 1,109 cases were 
completed for an unweighted completion rate of 86.0 percent. Table 3 shows the number of cases sampled 
for reinterview, the number of those that resulted in completed reinterviews, and the unweighted completion 
rate by participation status. The cooperation rate for AE participants was much higher than the rate for 
nonparticipants. 



Table 3. Participation status by reinterview sample sizes, completed interviews, and unweighted 
completion rates 



Participation status 


Sample size 


Completed reinterviews 


Unweighted completion 
rate 


All 


1,289 


1,109 


86.0% 


AE Participants 


917 


882 


96.2% 


AE Nonparticipants 


372 


227 


61.0% 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, 



1995. 



The main reason for not completing the reinterview was the refusal of the respondent to 
participate. Approximately 61 percent of nonrespondents in both participants and nonparticipants refused 
to participate in the reinterview. A majority of the remaining nonresponse was due to the ina bility of 
interviewers to contact the respondent during the reinterview time period. This reason had several causes, 
including never making contact with the respondent, reaching a disconnected telephone number or number 
that had been changed, and attempting to reach someone who had moved to a new household with no 
telephone or forwarding number. 

The reinterview was conducted using the same CATI system that was used in the original 
interview, modified to display the reinterview questions instead of the original ones. The interviewers read 
identical words to the same respondent who completed the original interview. No substitutions were 
allowed for the original respondent. 

After all of the questions for the reinterview were asked, the original and reinterview 
responses were compared automatically by the computer for a subset of questions. This is the 
reconciliation process discussed earlier. Up until this time, the interviewer was unaware of the responses 
given by the respondent in the original interview. For any of the selected questions that had different 



responses in the original and reinterview, a reconciliation screen appeared and the interviewer asked which 
of the two responses was the best answer. A typical screen used to resolve the differences is shown in 
Exhibit 1 . For most of the analysis, the results are based on differences between the original and the 
reinterview responses. When the differences between the original and reconciled responses are used, they 
are clearly identified. 



Exhibit 1. — Typical CATI reconciliation screen 



14.08 Rll 



One time we spoke with you we recorded that you did not participate in computer-based or 
interactive video instruction without an instructor present and one time we recorded that 
you did participate in computer-based instruction without an instructor present. What is 
the best answer? 

( ) 

1 PARTICIPATED IN COMPUTER-BASED INSTRUCTION ONLY 

2. PARTICIPATED IN INTERACTIVE VIDEO INSTRUCTION ONLY 

3. PARTICIPATED IN BOTH 

4 . DID NOT PARTICIPATE IN EITHER 

5 . BOTH ANSWERS CORRECT (PARTICIPATED AFTER ORIGINAL 
INTERVIEW WAS CONDUCTED OR 12 MONTHS PRIOR TO ORIGINAL 
INTERVIEW) 



3. Analysis Methods 

Several statistics have been developed to assess the reliability of reporting using 
reinterview data. The two statistics used in this report are the gross difference rate and the net difference 
rate. These statistics were used in previous NHES reinterview reports (Brick and West, 1992; Brick, 
Rizzo, and Wemimont, 1995) and are well documented in the reinterview literature (Hansen, Hurwitz, and 
Pritzker, 1964; Biemer and Stokes, 1991; US. Bureau of the Census, 1985). 

For dichotomous response variables, the gross difference rate measures the proportion of 
questions with different responses in the two administrations of the interview. Thus, it is an estimate of the 



reliability or consistency of reporting. The net difference rate takes account of offsetting misclassifications. 
If the second interview is the true value for the respondent, the net difference rate estimates the bias. 

Since the goal of this analysis is to provide estimates of response variability of the national 
estimates, the probability of selection for the interviews and the other weighting adjustments are included in 
the analysis. The sample weight associated with each completed interview was adjusted to account for the 
differential sampling of participants and nonparticipants employed in the reinterview. The weights for 
nonparticipants were multiplied by a factor of 2.5 to adjust for the differences in the sampling rates. The 
replicate weights used to compute the sampling errors of the estimates were also adjusted in this manner so 
that the appropriate sampling errors could be computed using jackknife methods. 

Table 4 shows the general format of the possible reporting outcomes from the original 
interviews and reinterviews when the question has only two possible values. From tables formatted in this 
fashion, it is possible to estimate several features of the consistency of the reporting between the original 
survey and the reinterview. For example, the off-diagonal cells estimate the responses that were reported 
differently in the original interview and the reinterview. The definitions of the statistics used in this report 
are given below, where the cell counts are the weighted totals. Cases with missing values for the 
characteristic are dropped from the analysis. 



Table 4. General format of interview-reinterview results 





Original interview 




Reinterview 


Number of 
cases with 
characteristic 


Number of 
cases without 
characteristic 


Total 


Number of cases 
with a characteristic 


a 


b 


a + b 


Number of cases 
without a characteristic 


c 


d 


c+d 


Total 


a + c 


b + d 


n=a+b+c+d 



3.1 



Gross Difference Rate 



The gross difference is equal to the weighted number of cases reported differently in the 
original interview and the reinterview. The gross difference rate is the weighted ratio of the gross 
difference divided by the estimated total number of cases. The gross difference rate is: 

G = — l — 1 1 *2,- } 2 (3.1) 

2 . 1 w i 

where xjj is the response to the original interview for case /; 

X 2 i is the response to the reinterview for case and 

w i is the full sample weight for case / described in the previous section. 

For characteristics that have exactly two possible outcomes, the gross difference rate, 
expressed as a percentage, can be written using the terms from Table 4 as 

G = 100^±£ (3.2) 

Yl 

This can easily be seen to be a special case of (3.1) where the Xj terms only take on the 

values of 0 or 1 . The gross difference rates for all questions were computed using (3 .2) and only data from 
the original and reinterview responses unless otherwise noted. For binary data, it is clear from (3.2) that 
the gross difference rate is an estimate of the percentage of cases not reported the same in both interviews, 
i.e., those falling in the off-diagonal cells. The gross difference rate divided by 2 is a measure of the 
response variance. Forsman and Schreiner (1991) show that this is an unbiased estimate of response 
variance if the observations are independent and identically distributed. The response variance is defined as 
the variation associated with the responses to the same question when the survey is repeated under the same 
general conditions. 

For nominal variables, neither (3.1) nor (3.2) can be used to compute the gross difference 
rate because the values assigned to the levels of the characteristic are not scaled. In this report, two 
approaches were taken to handle nominal variables, depending on the specific variable. For some nominal 
variables, response categories were collapsed so that a binary variable resulted and the gross difference rate 
was computed for this new binary variable using (3.2). This approach was used for all the questions about 
barriers to participating that had response categories of ‘major,’ ‘minor,’ or ‘no obstacle.’ The minor and 
no obstacle response categories were collapsed together. The response categories ‘major,’ and ‘minor’ 
were also collapsed to examine the response variability from a different perspective. The collapsing 



strategy was also used for the question about how interested adults were in taking a work-related course 
(WRHOWINT) by collapsing the ‘somewhat’ and ‘slightly’ interested response categories and the question 
about computer based or interactive video instruction (CVONLY) by collapsing the ‘computer instruction,’ 
‘interactive video instruction,’ and ‘both’ response categories. 

The second approach was used for three questions with multiple response categories. Two 
of the questions were about language background and had response categories of ‘English,’ ‘Spanish,’ or 
‘Another language.’ For these questions, three binary variables were computed from the original variable 
and then the gross difference rate was computed for each new variable using (3.2). The first b inar y 
variable has the value 1 if the response was ‘English’ and has the value 0 otherwise; the second b inar y 
variable has the value 1 if the response is ‘Spanish’ and has the value 0 otherwise; the third binary variable 
has the value 1 if the response is ‘Another language’ and has the value 0 otherwise. The other question was 
the one that asked which obstacle was the main thing that prevented the adult from participating (F23). For 
this question, three binary variables were created and the rate was computed using (3.2). The first b inar y 
variable has the value 1 if the response was ‘Time’ and has the value 0 otherwise; the second binary 
variable has the value 1 if the response was ‘Money or cost’ and has the value 0 otherwise; the third b inar y 
variable has the value 1 if the response was ‘Child care’ and has the value 0 otherwise. Note that none of 
the responses to this question was ‘Transportation’ so no binary variable was created for this level. The 
same procedure was followed for computing the net difference rate. 



3.2 



Net Difference Rate 



The net difference rate can be defined for characteristics that are binary or continuous. 
The net difference rate for a continuous variable is given by 



N = — —I - *2, } 

2. j w i 



(3.3) 



where the variables are defined as in (3.1). The net difference rate is thus the average difference between 
the original and reinterview responses. 

For the binary case, the net difference is the difference between the weighted number of 
cases with a characteristic as reported in the original interview and the weighted number of cases in the 
reinterview. That is, (a + c) - {a + b) = c - b, using the terms in Table 4. Thus, a positive net difference 
rate indicates that more adults reported having the characteristic in the original interview than in the 



reinterview. While the gross difference includes differences in both directions, the net difference is the 
nonoffsetting part of the gross difference. Written as a percentage, the net difference rate is: 

N= 100^- (3.4) 

n 

If the reinterview response is the "true" value, or at least a better approximation to the true 
value, then the net difference rate is a measure of the bias of the estimate. Generally speaking, this was not 
the case in the NHES:95 since the reinterview responses were collected under the same conditions as the 
original interview. Brick, et al. (1994) discuss this issue in more detail. In some surveys, it is assumed 
that reconciliation results in more accurate responses so that the net difference rate computed using the 
original and the reconciled responses is a valid estimate of response bias. Brick and West (1992) and 
Brick, et al. ( 1 994) found that there was little empirical support for this assumption, even for reconciled 
data. 



The net difference rate computed from the original and reinterview data can be used to 
evaluate one of the assumptions associated with the gross difference rate. If the reinterview is an 
independent replication of the original interview, then the gross difference rate is a valid measure of 
response variance. Generally, it is assumed that this condition holds, but the net difference rate provides a 
means of partially evaluating this assumption. If the interviews are replications, then the estimated net 
difference rate should be equal to zero in expectation (the original interview and reinterview should have the 
same average value). Biemer and Forsman (1992) discuss this issue more fully. Thus, the net difference 
rates for the questions in the AE reinterview study presented below can be used to evaluate this assumption. 

The Bias Study mentioned earlier was undertaken to attempt to more accurately measure 
response bias than could be done using these reinterview data. This study undertook to obtain responses 
from a subsample of respondents using very different methods than used in the reinterview. The data from 
this study provide additional insight on the response bias of the questions in the AE interview. 



4. Findings 

Below, the gross and net difference rates for the reinterview questions are presented along 
with a discussion of the implications of the results for the analysis of estimates from the AE component of 
the NHES:95 and the planning for future AE studies. 



The analysis is divided into three sections. The first part considers questions related to the 
estimation of the rates at which adults participate in education activities. All nine questions used to define 
AE participation and the type of AE participation were included in the reinterview and are discussed. 
Another question related to participation, but not included in its formal definition, is about using computer- 
based or interactive video instruction without an instructor being present. This question is covered with the 
other participation questions. 

The second part covers questions related to the education background and work force 
activities of the adult. These are questions about years of schooling, degrees obtained, English speaking 
ability, labor force status, and job benefits. The third and final part covers questions that were asked about 
obstacles that may have prevented adults from taking part in AE activities, the number of programs or 
courses, and other characteristics of participation in AE activities. The questions about obstacles are 
sometimes called the barriers questions. 

For each of the parts, the estimates and their standard errors are presented along with the 
gross and net difference rates and their standard errors. The estimates are the weighted percent of 
respondents reporting in the first response category of the question based only on the data from the 
respondents to the reinterview. Since these are restricted to the reinterview subsample, the estimates may 
differ from those from the full sample. The estimates are restricted to those adults who were asked the 
question. For example, persons whose main language was English were not asked the questions about 
participating in English as a Second Language (ESL). The statistics reported (the percent of adults taking 
ESL and the gross and net difference rate) for this question were based on the subset of respondents whose 
main language was not English. Equation (3.2) was used for the gross difference rate and equation (3.4) 
was used for the net difference rate unless specified otherwise. 

Questions that were answered by fewer than 20 adults in the reinterview are excluded from 
the analysis because of the instability of estimates based on such small sample sizes. In addition, a short 
series of questions asked about a specific work-related course are excluded because of the difficulty 
associated with matching the course described in the reinterview to the exact same course in the original 
interview. 



In general, the standard errors for the gross and net difference rates are large and not very 
stable since they too are estimates based on relatively small sample sizes. Thus, the criterion that the error 



measures be statistically different than zero using a significance level of 10 percent 2 is not satisfied for 
some questions that have large gross difference rates. For these questions, the large rates are mentioned but 
qualified by noting that they do not meet the statistical criterion. 



4.1 Questions About AE Participant Rates 

Table 5 shows the estimates, the gross and net difference rates, and the standard errors for 
the questions that define AE participation. The nine questions that are used to classify an adult as an AE 
participant are presented along with the overall participation composite variable (an adult is a participant if 
he/she was involved in any of the nine specific activities). The last question in the table is about using 
computer-based or interactive video instruction without an instructor being present. The variable names 
and numbers in the table can be referenced to the specific questions by looking at the reinterview 
questionnaire in the appendix. For example, ESLANG is questionnaire number B1 that asks about 
participation in ESL. 

The gross difference rates for all of the participation questions are relatively small. The 
only gross difference rate that is statistically greater than zero is for the overall participation variable, and 
the gross difference rate for this statistic indicates that only about 13 percent of the adults were inconsistent 
in their reporting of questions used to construct this variable (i.e., reported conflicting responses in the 
original interviews and reinterviews). The main contributors to this inconsistency are the questions that 
measured participation in work-related (WRACTY) and personal development courses (SAACTY). These 
questions have gross difference rates that are roughly equivalent to the rate for the overall participation 
variable. 



2 The 10 percent significance test corresponding to a t-statistic of 1.65 is used in this analysis without taking into 
account the problem of testing many statistics at the same time. This approach is followed because of the 
exploratory nature of the analysis. One of the main goals of the study was to identify questions that can be 
improved and the loss associated by erring in suggesting that more questions be studied is not as large as the 
opposite type of error. 



Table 5. Participation questions by estimated percent, gross and net difference rates based on 
unreconciled reinterview responses 



Question 


Sample size 


Estimated 


Gross 

difference rate 


Net 

difference rate 


percent 


s.e. 


estimate 


s.e. 


estimate 


s.e. 


Overall participation 


1,109 


44.9 


142.3 


13.4 


3.7 


-2.7 


37.8 


ESLANG (Bl) 


40 


6.5 


23.6 


0.0 


0.0 f 


0.0 


0.0 f 


BSIMPROV (Cl. a)) 


146 


3.8 


14.0 


3.6 


13.5 


1.3 


5.1 


BSGED(Cl.b) 


146 


4.3 


16.1 


2.3 


8.7 


1.8 


6.8 


BSHEQUV (Cl.c) 


146 


1.5 


5.7 


1.2 


4.6 


0.3 


1.0 


CRDEGREE (Dl .a) 


1,109 


12.0 


36.2 


2.8 


5.5 


0.6 


4.6 


CRVOCDIP (Dl.b) 


1,109 


3.9 


11.9 


4.0 


10.2 


0.5 


3.3 


APPRENTI (El) 


1,109 


1.2 


3.7 


1.7 


2.6 


-0.4 


1.3 


WRACTY (FI) 


1,109 


20.7 


62.3 


12.5 


15.2 


-4.7 


8.6 


SAACTY (Gl) 


1,109 


22.4 


67.5 


14.3 


20.7 


0.3 


23.3 


CVONLY (HI) 


923 


17.0 


8.4 


12.8 


9.4 


1.8 


2.1 



T 



The estimate of the standard error is zero, but the estimate is subject to variation due to sampling error. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, 
1995. 



The net difference rates in Table 5 are ones based on the comparison of the original and 
reinterview values. Since none of these questions have statistically significant net difference rates, the 
estimates are consistent with the assumption that the reinterview was an independent replication of the 
original interview, at least for these questions. Thus, the assumption that the gross difference rate is a valid 
measure of response variability is supported by these results. 



Table 6 presents the statistics on the same questions, but the gross and net difference rates 
are computed using the reconciled reinterview responses rather than the unreconciled ones used in Table 5. 
The reinterview response was considered to be the reconciled value unless the respondent indicated in 
reconciliation that this was an incorrect response. The estimates of primary interest in this table are the net 
difference rates. If the reconciliation resulted in obtaining more reliable responses for these questions, then 
the reconciled net difference rates should be more appropriate estimates of the bias in the estimates. 
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Table 6. Participation questions by estimated percent, gross and net difference rates based on reconciled 
reinterview responses 



Question 


Sample size 


Estimated 


Gross difference 
rate 


Net 

difference rate 


percent 


s.e. 


estimate 


s.e. 


estimate 


s.e. 


Overall participation 


1,109 


44.9 


142.3 


12.5 


10.0 


-3.4 


37.3 


ESLANG (Bl) 


40 


6.5 


23.6 


0.0 t 


o.o* 


0.0 


o.of 


BSIMPROV (Cl. a)) 


146 


3.8 


14.0 


2.3 


8.4 


0.6 


2.3 


BSGED (Cl.b) 


146 


4.3 


16.1 


1.9 


6.9 


1.4 


5.2 


BSHEQUV (Cl.c) 


146 


1.5 


5.7 


1.0 


3.7 


0.1 


0.4 


CRDEGREE (Dl .a) 


1,109 


12.0 


36.2 


1.7 


5.0 


0.9 


2.7 


CRVOCDIP (Dl.b) 


1,109 


3.9 


11.9 


2.8 


6.6 


0.6 


3.6 


APPRENTI (El) 


1,109 


1.2 


3.7 


1.3 


1.4 


-0.1 


2.1 


WRACTY (FI) 


1,109 


20.7 


62.3 


9.7 


7.4 


-5.7 


4.7 


SAACTY(Gl) 


1,109 


22.4 


67.5 


9.4 


12.3 


-1.2 


12.5 


CVONLY (HI) 

t TT-k« + l . j _ « 


923 


17.0 


8.4 


10.0 


4.2 


1.1 


1.4 



f The estimate of the standard error is zero, but the estimate is subject to variation due to sampling error. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey 



None of the net difference rates from the reconciled responses are significantly different 
from zero, at least partially due to the large sampling errors of these statistics. For the three variables with 
larger gross difference rates observed in Table 5 (participation overall, in work-related activities, and in 
personal development courses), the net differences are negative. This means that more adults reported 
participating in the reconciled reinterview than in the ori ginal interview. 

Although not statistically significant, these results are consistent with the hypothesis that 
adults might be more likely to be unable to recall some type of participation in the original interview, but 
report more in the reconciled reinterview. In particular, one theory is that the original interview might act 
as a means to either improve recall or to help the respondents better understand how their activities fit with 
the survey definitions. The results are far from conclusive concerning this hypothesis, but they do suggest 
that adults may have under-reported AE participation, particularly work-related courses, in the NHES:95. 

One of the main reasons for conducting the Bias Study mentioned earlier was the concern 
that the reconciliation responses would not be much better than the reinterview responses in terms of 
accurately corresponding to the activities of adults. The fact that the reconciled and unreconciled net 



» 

difference rates are not very different could be a reflection that this concern is valid. If so, then the 
reconciled net difference rates are not valid estimates of response bias. The Bias Study, which 
* concentrated on the bias in the estimates of work-related and personal development course participation, 

provides additional information on this topic. 

Overall, the measurement error statistics for the participation questions are relatively small 
® and the adults consistently responded to these questions across interviews. Since these questions are so 

critical to the AE interview, it might be worthwhile studying the work-related and personal development 
courses questions to determine if there are other ways to improve them. These questions were recognized 
as being the most difficult ones during the development of the interview and continued efforts to improve 
® them appear to be worthwhile. The results also suggest that adults may be under-reporting participation, 

particularly in work-related courses, but the reinterview and the reconciliation of responses do not 
adequately address this potential bias. 



4.2 Questions About Background 

Table 7 gives the estimates, the gross and net difference rates, and the standard errors for 
questions related to the education background and work force activities of the adult. The questions are 
ones asked either at the beginning or end of the interview. Two questions (MEDICAL and SICKPAY) 
correspond to answers to question numbers 128 or 129, depending on whether or not the person was self- 
employed. Equations (3.1) and (3.3) were used to compute the gross and net difference rates for the 
following continuous questions: IBGRADE, IBEMPL12, and IBWORKMO. 

The gross difference rates for all 23 background questions are small, with only three 
questions having a gross difference rate over 10 percent and none over 20 percent. The three questions 
with rates over 10 percent are ones about earning a vocational or technical diploma after leaving high 
school (IBVOCDIP), whether their occupation had legal or professional requirements for continuing 
training or education (REQUIRMN), and whether or not they had been unemployed and looking for work 
for as long as a month during the last year (UNEMLOOK). Of these three, only REQUIRMN is 
statistically significant. 

The net difference rates for two questions appear to be large (IBVOCDIP and 
IBOTHEMP), but the standard errors of both of these rates are so large that the estimates are not 
significantly different from zero. On the other hand, two questions (IBSPEAK9 1 and REQUIRMN) have 
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relatively small net difference rates, but are significantly different from zero. These results may have more 
to do with the instability of the standard errors of the net difference rates than any bias in the estimates. 

Only one of these questions (IBGED) was reconciled if the respondent gave different 
responses in the original interview and reinterview. The net difference rate for IBGED is -0.4 percent (with 
a standard error of 7.9 percent). This is not very different from the unreconciled rate and does not reveal 
any potential biases associated with this question. 



Table 7 . Background questions by estimated percent, gross and net difference rates based on unreconciled 
reinterview responses 



Question 


Sample size 


Estimated 


Gross difference 
rate 


Net 

difference rate 


percent 


s.e. 


estimate 


s.e. 


estimate 


s.e. 


IBGRADE (Al) 


1,107 


6.0 


2.9 


1.2 


0.5 


0.1 


0.1 


IBVOCDIP (A10V) 


253 


13.3 


14.1 


17.0 


10.9 


-8.4 


29.9 


IBDIPL (A2) 


476 


67.9 


69.8 


0.9 


2.8 


0.8 


2.6 


IBUSDIPL (A3) 


651 


96.6 


0.8 


4.0 


10.2 


0.5 


3.3 


IBDIPLYR (A4) 


651 


4.0 


3.9 


2.0 


3.9 


-0.3 


0.2 


IBGED (A5) 


1,024 


7.3 


20.6 


4.9 


0.7 


-1.8 


9.4 


IBWORK12 (A6) 


1,109 


74.6 


23.2 


5.2 


7.1 


-0.8 


0.6 


IBSELFEM (A7) 


923 


18.2 


13.1 


3.9 


2.0 


0.3 


5.9 


IBOTHEMP (A8) 


153 


40.4 


27.9 


6.3 


11.7 


6.3 


11.7 


IBEMPL12 (A9) 


906 


1.4 


0.2 


0.3 


0.2 


0.0 


0.0 


IBLANG1 (AlO.a) 


1,109 


91.1 


3.6 


1.0 


0.7 


-0.4 


1.2 


IBLANG2 (AlO.b) 


1,109 


2.8 


0.2 


0.3 


1.0 


0.1 


0.4 


IBLANG91 (AlO.c) 


1,109 


6.1 


3.7 


0.7 


0.3 


0.2 


1.7 


IBSPEAK1 (All. a) 


1,109 


96.0 


5.2 


1.1 


1.2 


-1.0 


1.6 


IBSPEAK2 (Al 1 .b) 


1,109 


1.6 


1.9 


0.7 


1.7 


0.4 


1.7 


IBSPEAK91 (All.c) 


1,109 


2.4 


3.3 


0.6 


0.1 


0.6 


0.1 


REQUIRMN (114) 


1,109 


21.5 


25.2 


14.7 


6.0 


1.5 


0.5 


IBWORKMO (124) 


915 


10.7 


0.2 


4.7 


4.2 


0.2 


0.6 


UNEMLOOK (125) 


244 


36.8 


1.5 


15.6 


19.5 


-0.8 


5.2 


MEDICAL (I28/9.a) 


829 


72.3 


10.8 


7.6 


1.3 


-2.7 


10.9 


SICKPAY (I28/9.b) 


759 


65.4 


18.5 


7.6 


3.5 


-0.8 


3.2 


VACATPAY (128. c) 


759 


74.7 


2.8 


6.4 


3.0 


1.9 


3.2 


RETIRMNT (I28.d) 


829 


61.1 


12.9 


8.6 


9.8 


-1.7 


3.7 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey 
1995. 



The background questions are generally specific questions that the respondent apparently 
comprehends and can respond to consistently from interview to interview. The evidence from the 
reinterview shows that there are no serious response problems with any of the questions. Only three 
questions have gross difference rates that even merit further consideration and the gross difference rates for 
these questions are not large. However, some improvement in the question wording for these three could be 
considered. For example, the unemployment question (UNEMLOOK) is a compound question (both 
unemployed and looking for work) qualified by a time frame, and it may not be clear to the respondent 
whether the descriptor (for as long as a month) applies to either or both parts of the question. 



4.3 Questions About Barriers and Other Characteristics 

Table 8 gives the measurement error estimates for barriers questions (questions about 
obstacles that may have prevented adults from taking part in AE activities) and other characteristics of 
participating in AE activities. Equations (3.1) and (3.3) were used to compute the gross and net difference 
rates for the following continuous questions: CRPRGNUM, WRNUM, SANUM, and CVNUM. The 
gross difference rates in Table 8 are much larger than those discussed earlier. Approximately one-third of 
the 3 1 questions have gross difference rates greater than 20 percent, another third have gross difference 
rates between 10 percent and 20 percent, and the remaining third have gross difference rates of less than 10 
percent. 



The vast majority of the questions with large gross difference rates are from the barriers 
questions (questions in F22, F23, and F24). For F22 and F24, respondents were asked if each of a list of 
potential obstacles was a major, minor, or no obstacle. For this analysis, the minor and no obstacle 
responses are considered to be the same response. A different collapsing of the response categories is 
presented later. 



Most of the items in the list of potential obstacles in F22 and F24 have large gross 
difference rates: only 4 of the 15 questions on the lists that were analyzed had gross difference rates less 
than 10 percent and the largest gross difference rate approaches 50 percent (WRTIWORK). As with other 
gross and net difference rates discussed above, nearly all of these estimates are not statistically significant 
different from zero because of the large standard errors. 



Table 8. Barriers questions and other characteristics by estimated percent, gross and net difference rates 
based on unreconciled reinterview responses 



Question 


Sample size 


Estimated 


Gross difference 
rate 


Net 

difference rate 


percent 


s.e. 


estimate 


s.e. 


estimate 


s.e. 


CRPRGNUM (D2) 


249 


1.2 


0.1 


0.9 


0.5 


0.1 


0.1 


CRAWARE (D17) 


245 


86.1 


2.6 


13.8 


8.4 


13.8 


8.4 


CREMPREQ (D18.a) 


213 


5.1 


1.5 


5.0 


3.9 


-5.0 


3.9 


CREMPWP (D18.b) 


213 


42.4 


3.9 


22.9 


11.5 


16.5 


12.3 


CREMPSPA (D18.c) 


213 


9.6 


2.1 


10.2 


36.9 


10.2 


10.5 


CREMPPAY (D18.d) 


213 


28.5 


3.1 


17.3 


11.5 


17.3 


11.5 


WRNTJM (F2) 


348 


2.5 


0.1 


2.4 


0.4 


0.1 


0.1 


WRINTRST (FI 9) 


685 


24.6 


13.7 


16.0 


5.0 


-2.2 


2.4 


WRHOWINT (F20) 


212 


57.1 


12.3 


15.7 


1.5 


15.7 


1.5 


WRKNOW (F21) 


212 


64.7 


14.4 


26.8 


17.1 


5.1 


18.7 


WRPRTIME (F22.a) 


144 


50.7 


3.1 


30.1 


40.7 


-4.8 


24.9 


WRPRCOST (F22.b) 


144 


42.5 


3.0 


6.5 


19.3 


3.4 


10.1 


WRPRCHIL (F22.c) 


56 


52.6 


18.5 


39.4 


66.8 


39.4 


66.8 


WRPRTRAN (F22.d) 


144 


10.2 


2.2 


5.1 


15.1 


2.8 


8.5 


WRPRGEN1 (F23, time) 


43 


47.7 


9.0 


8.0 


4.9 


-5.3 


5.3 


WRPRGEN2 (F23, money) 


43 


44.1 


9.5 


13.6 


5.5 


2.1 


6.6 


WRPRGEN3 (F23, child care) 


43 


8.3 


5.3 


5.6 


3.5 


3.3 


3.4 


WRTIFAM (F24A.a) 


58 


30.0 


4.5 


4.2 


5.7 


4.2 


5.7 


WRTICHOR (F24A.b) 


58 


34.9 


28.5 


16.8 


18.9 


6.0 


9.7 


WRTICLHR (F24A.C) 


56 


50.0 


23.0 


30.8 


35.6 


- 7.8 


17.3 


WRTIWORK (F24A.d) 


56 


47.5 


38.2 


49.3 


54.8 


-9.4 


23.0 


WRTIACTI (F24A.e) 


58 


17.5 


24.1 


14.3 


90.4 


9.2 


95.7 


WRTITRAV (F24A.f) 


58 


18.3 


2.7 


24.4 


28.3 


13.9 


19.8 


WRTIOTH (F24A.g) 


58 


5.4 


5.9 


14.9 


17.2 


-4.9 


9.2 


WRMOTUIT (F24B.a) 


45 


73.3 


27.7 


20.5 


6.8 


-12.8 


29.3 


WRMOBOOK (F24B.b) 


45 


54.2 


4.3 


36.2 


44.2 


-22.1 


7.9 


WRMOTRAN (F24B.d) 


45 


2.7 


8.5 


0.0 


0.0 T 


0.0 


0.0 T 


WRMOOTH (F24B.e) 


45 


13.9 


43.4 


31.5 


97.0 


31.5 


97.0 


WREMPOFF (F26) 


466 


18.1 


8.9 


12.8 


1.5 


3.9 


9.3 


SANUM (G2) 


258 


1.9 


0.1 


3.0 


1.0 


0.1 


0.1 


CVNUM (H2) 


116 


4.4 


4.3 


10.7 


7.3 


0.3 


0.9 



T The estimate of the standard error is zero, but the estimate is subject to variation due to sampling error 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey 
1995 . 
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In order to be asked these questions, the adult had to satisfy three criteria: (1) not be a 
participant in work-related courses; (2) be interested in taking one or more work-related courses, and; (3) 
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know of a course he/she could take. As a result, only about one-fifth of the AE respondents were 
administered these work-related barriers questions (similar questions in other sections of the interview did 
not have enough respondents to meet the minimum reporting requirements of this report). Nevertheless, the 
point estimates of the gross difference rates for these questions are larger than any of those observed for 
other questions. 



As mentioned earlier, the ‘minor’ and ‘no obstacle’ response categories were collapsed 
together to form a binary variable for the analysis described above. An alternative approach is to collapse 
‘major’ and ‘minor’ together to form the new binary variable. The original collapsing addresses the 
question of how consistently adults classified an obstacle as being major, while the alternative collapsing 
focuses on whether or not it was an obstacle, regardless of the intensity of the obstacle. Table 9 gives the 
estimated gross difference rates and standard errors for the two versions of the binary variable. Many of 
the very large gross difference rates appearing in the original collapsing of the response categories are 
smaller when ‘major’ and ‘minor’ are collapsed, although for one question (WRTIACTI) the opposite is 
true. Adults may be more consistent in reporting whether or not it was an obstacle, but have more 
difficulty assessing whether or not the obstacle is major or minor, even though this is not conclusive given 
the size of the standard errors for these estimates. This suggests a somewhat better strategy for analysis of 
these questions might be to ignore the ‘major’ and ‘minor’ categories and analyze only whether or not the 
adult reported it as an obstacle. 

Another question with a large gross difference rate is the one that asked adults if they knew 
of any work-related courses that they could have taken (WRKNOW), but the standard error of the estimate 
is so large that even this large estimate is not statistically significant. This question is very important 
because it is part of a series of questions used in screening before asking the barriers questions. The gross 
difference rates for the other two questions in this series (WRINTRST and WRHOWINT) are also 
moderately large. These results suggest that the questions used to determine which adults are asked the 
barriers questions may be subject to response problems in addition to the problems in the barriers questions 
themselves. 



Turning briefly to the net difference rates, only 2 of the 31 questions in Table 8 have net 
difference rates that are statistically significant at the 10 percent significance level (WRHOWINT and 
WRMOBOOK). This again reinforces the assumption that the reinterview responses can be treated as 
independent repetitions of the interview. 

Taken as a whole, the findings suggest that the barriers questions and other questions 
related to them may be subject to important response variability. The size of the standard errors of the 



estimates limits the ability to determine if the estimates are statistically significant, but the point estimates 
for many of these questions are large. In this small sample, the adults frequently responded differently 
when the same questions were asked in the original interview and the reinterview. The inconsistency of the 
responses to these questions should be considered when using these questions in analyses or reports. 
Furthermore, the method for addressing the concept of barriers to participating in AE activities, and the 
feasibility of identifying a reliable approach to do so, should be re-evaluated for future surveys. 



Table 9. Barriers questions by estimated gross difference rates for different methods of collapsing 
responses based on unreconciled reinterview responses 



Question 


Collapsing 
minor and none 


Collapsing 
major and minor 


estimate 


s.e. 


estimate 


s.e. 


WRPRTIME (F22.a) 


30.1 


40.7 


9.6 


28.1 


WRPRCOST (F22.b) 


6.5 


19.3 


10.8 


31.9 


WRPRCHIL (F22.c) 


39.4 


66.8 


7.4 


16.8 


WRPRTRAN (F22.d) 


5.1 


15.1 


20.0 


58.9 


WRTIFAM (F24A.a) 


4.2 


5.7 


6.2 


8.5 


WRTICHOR (F24A.b) 


16.8 


18.9 


6.0 


8.1 


WRTICLHR (F24A.C) 


30.8 


35.6 


5.0 


6.9 


WRTIWORK (F24A.d) 


49.3 


54.8 


33.3 


38.0 


WRTIACT1 (F24A.e) 


14.3 


90.4 


41.3 


44.9 


WRTITRAV (F24A.f) 


24.4 


28.3 


18.2 


20.5 


WRTIOTH (F24A.g) 


14.9 


17.2 


15.8 


18.3 


WRMOTUIT (F24B.a) 


20.5 


6.8 


_t 


_t 


WRMOBOOK (F24B.b) 


36.2 


44.2 


0.0 


0 . 0 ” 


WRMOTRAN (F24B.d) 


0.0 


O.O 11 


8.7 


26.7 


WRMOOTH (F24B.e) 


31.5 


97.0 


31.5 


97.0 



T The estimate is undefined because the sample in the ‘no obstacle’ cell is zero. 



^ The estimate of the standard error is zero, but the estimate is subject to variation due to sampling error. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey, 
1995 . 
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Conclusion 



The Adult Education reinterview for the NHES:95 has been used to examine how 
consistently adults responded when asked the same questions on two occasions. The gross difference rate 
was used to measure this consistency. 

For most of the questions, the reporting was consistent and only a few modifications need 
to be considered for future surveys. However, for questions associated with barriers to AE participation, 
more problems were identified. The important findings and implications are summarized below, followed 
by some discussions of the methodology of the reinterview. 

For questions used to determine whether or not the adult participated in adult education 
during the last 12 months, the adults responded consistently. The gross difference rates for these questions 
are either small or moderate. The two questions with moderate, but not statistically significant, gross 
difference rates were ones that asked about taking work-related activities and personal development 
courses. Some minor modifications to these two questions could be considered to improve the consistency 
of these questions and of the overall participation rate. More importantly, the Bias Study finding s suggest 
that respondents may not report all their work-related and personal development courses and this may spur 
changes in these questions. 

For questions about the education background and labor force history of the adults, the 
adults also gave very consistent responses. The respondents apparently had a good understanding of nearly 
all of these questions, and they gave the same answers in both the interview and the reinterview. Three 
questions in this set (questions about acquiring a vocational diploma, whether they had been unemployed 
and looking for work, and whether their occupation had legal requirements for continuing training) were 
identified that should be reviewed before future surveys. 

For questions about barriers that may have prevented adults from participating in adult 
education activities, the adults generally did not respond as consistently. Most of the barriers questions had 
either high or moderate gross difference rates, although most of the estimates were not statistically 
significant. One approach to limit these response problems is to avoid using measures of intensity (major 
or minor) when analyzing barriers since there is some evidence to suggest respondents might be more 
consistent in reporting whether or not it was an obstacle than they were in reporting whether it was a major 
or minor obstacle. Analysts should consider these apparent response problems when using these questions. 
Other precautions, such as making sure any differences are large before assigning them much significance, 
should also be considered. 



Given the response problems associated with the questions on barriers, it may be useful to 
reconsider the method of addressing obstacles. For example, given the difficulty associated with 
discriminating between major and minor obstacles, this difference might be dropped. Other types of 
questions about behaviors of adults might also be investigated rather than the current questions. For 
example, rather than asking directly about barriers to participation, questions could be asked about whether 
adults took any steps to try to either take courses or obtain information about courses and, if so, whether 
they were discouraged for specific reasons. 

Methodologically, the AE reinterview served its major purposes of identifying questions 
with high error rates and providing feedback to help improve the design of the questions for future surveys. 
However, the standard errors of the gross difference rates were often too large to adequately quantify the 
magnitude of the gross difference rates. Some large gross difference rates were not statistically significant 
because of these large standard errors. Several options are available to improve the precision of the 
estimates in future studies, the most obvious being to increase the sample size of the reinterview. Other 
approaches that should be considered are avoiding oversampling specific groups in the reinterview 
(participants were oversampled in this study) and using model-based methods (such as ignoring the 
sampling weights) to estimate the measurement error statistics rather than the full design-based estimates 
used in this study. 

The reconciliation of the responses to different responses to the original interview and the 
reinterview for a subset of questions did not provide much information about the response bias for the 
questions. The net difference rates computed from the reconciled responses were not very different from 
those computed using the unreconciled responses. The primary value of the unreconciled net difference 
rates was to evaluate whether or not the responses to the reinterview could be considered an independent 
replication of the original interview. In general, the unreconciled net difference rates supported this 
assumption. 



For future studies, the reconciliation of differences between the original interview and the 
reinterview should be re-evaluated. This study and the NHES:93 reinterview study found no evidence that 
the reconciled responses provide valid estimates of response bias. Other methods, such as the Bias Study 
or record checks studies when they are feasible, are better mechanisms for examining bias. 
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APPENDIX 



ADULT EDUCATION REINTERVIEW QUESTIONNAIRE 



NHES:95 Adult Education Reinterview 



INTR01. Hello, this is (interviewer) calling for the U.S. Department of Education. Recently, you 

completed an interview for a U.S. Department of Education study about participation in 
various kinds of educational activities. For quality control purposes, we are calling back 
a small number of people to reask a subset of the questions. This usually takes about 5 
minutes. 

INTR02. First, I have a few questions about your education, your work experience, and your 

language background. 

A. Initial Background 



A1 . What is the highest grade or year of school that you completed? 



IBGRADE 



up to 8th grade 1 

9th to 11th grade 2 

1 2th grade but no diploma 3 

HIGH SCHOOL DIPLOMA/EQUIVALENT 4 

VOC/TECH PROGRAM AFTER HIGH SCHOOL BUT NO VOC/ 

TECH DIPLOMA 5 

VOC/TECH DIPLOMA AFTER HIGH SCHOOL 6 

SOME COLLEGE BUT NO DEGREE 7 

ASSOCIATE'S DEGREE 8 

BACHELOR'S DEGREE 9 

GRADUATE OR PROFESSIONAL SCHOOL BUT NO DEGREE . 1 0 

MASTER'S DEGREE (MA, MS) 11 

DOCTORATE (PHD, EDD) 12 

PROFESSIONAL DEGREE BEYOND BACHELOR'S DEGREE 
(MEDICINE/MD; DENTISTRY/DDS; LAW/JD/LLB; ETC.) . 1 3 

REFUSED -7 

DON'T KNOW -8 



(ENTER ACTUAL GRADE, 
GO TO A2) 

(ENTER ACTUAL GRADE, 
GO TO A2) 

(GO TO A2) 

(GO TO A3) 

(GO TO A2) 

(GO TO A2) 

(GO TO A10V) 

(GO TO A2) 

(GO TO A5) 

(GO TO A5) 

(GO TO A5) 

(GO TO A5) 

(GO TO A5) 

(GO TO A2) 

(GO TO A2) 



A10V. Did you earn a vocational or technical diploma after leaving high school? 



IBVOCDIP YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 

A2. Do you have a high school diploma or its equivalent, such as a GED? 

IBDIPL YES 1 (GO TO A3) 

NO 2 (GO TO A6) 

REFUSED -7 (GO TO A6) 

DON'T KNOW -8 (GO TO A6) 



NOTE: 



Response categories shown in mixed cases (upper and lower) are read to the respondent by the interviewer. Those shown 
in all upper case are not read. 



A3. 



Did you receive your high school diploma or its equivalent in the U.S.? 



IBUSDIPL 



A4. 



IBDIPLYR 



A5. 



YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 

Did you receive your high school diploma or its equivalent in the past 12 months? 

yes 1 

no 2 

REFUSED -7 

DON'T KNOW -8 



Did you complete your high school requirements through a GED test? 



IBGED yes 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



A6. Did you work at a job for pay or income at any time in the past 1 2 months? 



IBWORK1 2 YES 1 (GO TO A7) 

N0 2 (GOTOA10) 

REFUSED -7 (GOTOA10) 

DON'T KNOW -8 (GOTOA10) 



A7. Were you self-employed at any time in the past 12 months? 



IBSELFEM YES 1 (go to A8) 

NO 2 (GO TO A9) 

REFUSED -7 (GO TO A9) 

DON'T KNOW -8 (GO TO A9) 



A8. Did you also work for another employer in the past 1 2 months? 



IBOTHEMP YES 1 (go TO A9) 

NO 2 (GOTOA10) 

REFUSED -7 (GOTOA10) 

DON'T KNOW -8 (GO TO AIO) 



If A7 = 1 and A8 NE 1 (self-employed only), 
then autocode A9 (# of employers) = 1 . 



A9. (Counting your self-employment,) For how many different employers did you work 

past 12 months? 



IBEMPL1 2 NUMBER □□ 

REFUSED -7 

DON'T KNOW -8 
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A10. Now, about your language background. What was the first language you learned to 

speak? 



IBLANG ENGLISH 1 (go TO INTR03) 

SPANISH 2 (GO TO All) 

ANOTHER LANGUAGE 91 (GO TO All) 

IBLANGOS SPECIFY 

REFUSED -7 (GO TO All) 

DON'T KNOW -8 (GO TO A1 1 ) 



Autocode A1 1 = 1 if AIO = 1. 



A1 1 . What language do you speak most at home now? 



IBSPEAK ENGLISH 1 

SPANISH 2 

(ANOTHER LANGUAGE SPECIFIED IN AIO) 3 

ANOTHER LANGUAGE 91 

IBSPEAOS SPECIFY 

REFUSED -7 

DON'T KNOW -8 



INTR03. Now, I'd like to ask you about different kinds of education and training programs, courses, 

workshops, and seminars you may have taken during the oast 12 months . (Please don't 
include day-time high school programs.) 



Ask B1 if A1 1 NE 1 (main language is other than English). 
Else, go to box before Cl. 



B. English as a Second Language 



B1. 



ESLANG 



These first questions are about English as a Second Language only . Please do not include 
other classes here. During the past 12 months, did you have a tutor or take any classes 
to learn English as a Second Language? 



yes 1 (gotoBIOV) 

NO 2 (GO TO BOX BEFORE Cl) 

REFUSED -7 (GO TO BOX BEFORE Cl) 

DON'T KNOW -8 (GO TO BOX BEFORE Cl) 






» 



For Participants 

BIOV. Is this ESL class a part of a college program? 



ESCOLL yes 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



O 
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If B10V = 1 , ask B6 and B7; then go to box before Cl. 



B3. What was the main reason you took English as a Second Language classes? 

[PROBE: READ LIKELY ANSWER(S)] 



ESREASON TO IMPROVE, ADVANCE, OR KEEP UP TO DATE 

ON CURRENT JOB 1 

TO TRAIN FOR A NEW JOB OR A NEW CAREER 2 

TO IMPROVE YOUR BASIC READING, WRITING, 

OR MATH SKILLS 3 

TO MEET A REQUIREMENT FOR A DIPLOMA, DEGREE, OR 

CERTIFICATE OF COMPLETION 4 

ESREAOS1 SPECIFY 

A PERSONAL, FAMILY, OR SOCIAL REASON 5 

TO IMPROVE COMMUNICATION SKILLS 6 

SOME OTHER MAIN REASON 91 

ESREAOS2 SPECIFY 

REFUSED -7 

DON'T KNOW -8 



B6. In the past 12 months , how many weeks did you attend ESL classes? 



ESWHEN NUMBER 



□ □ 



ESWHENUN 



ESWHENOS 



Unit 

DAYS 1 

WEEKS 2 

MONTHS 3 

SEMESTER 4 

QUARTER 5 

OTHER 91 

SPECIFY 

REFUSED -7 

DON'T KNOW -8 



(GO 


TO 


B7) 


(GO 


TO 


B7) 


(GO 


TO 


B7) 


(GO 


TO 


B7) 


(GO 


TO 


B7) 


(GO 


TO 


B60V) 


(GO 


TO 


B7) 


(GO 


TO 


B7) 



B60V. How many weeks was that? 

ESWEEK weeks □□ 



Collect number; autocode unit. 



B7. For about how many hours ( per dav/per week ) did you attend during the time you 

going to ESL classes? 



ESHRS NUMBER □□ 

Unit 

ESHRSUNT PER DAY 1 

PER WEEK 2 

REFUSED -7 

DON'T KNOW -8 
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B8. In the past 1 2 months , about how much of your own money would you estimate you paid 

for tuition, books, transportation, child care, and other expenses to take ESL classes? 



ESTUITON AMOUNT $□□,□□□ 

REFUSED -7 

DON'T KNOW -8 



If A 6 NE 1 (not worked in the past 12 months ), 
then go to box before Cl. 

If A7 = 1 and A8 NE 1 (self-employed only), 
then go to box before Cl. 



B14. (Not counting your self-employment,) Was your employer aware that you were taking or 

took the ESL class? 



ESAWARE yes 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



If B14 NE 1 (employer was not aware), 
then go to box before Cl. 



B15. 



Did your employer... 



ESEMPREQ a. 
ESEMPWP b. 
ESEMPSPA c. 
ESEMPPAY d. 



YES NO 

Require you to take these classes? 1 2 

Give you time off from work with or without pay? . 1 2 

Provide classroom space? 1 2 

Pay all or part of the cost, including tuition, books, 
and other costs like transportation? 1 2 



R 


DK 


-7 


-8 


-7 


-8 


-7 


-8 


-7 


-8 



Ask Cl if A2 NE 1 (no high school diploma) or 
A4 = 1 (received high school diploma 
in the last 12 months). 

If A3 NE 1 (foreign high school diploma), ask Cl. 



C. Basic Skills and GED Preparation 

Cl. Not including regular day-time high school classes, during the past 12 months , did you 

have a tutor or take any classes: 









YES 


NO 


R 


DK 


BSIMPROV 


a. 


To improve your basic reading, writing, and math 














skills? 




2 


-7 


-8 


BSGED 


b. 


To prepare to take the General Educational 














Development, or GED? 


. 1 


2 


-7 


-8 


BSHSEQUV 


c. 


In some other high school equivalency program 














or adult high school program? 


. 1 


2 


-7 


-8 
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If any Cl a, b, or c = 1, then ask C2. 
If A3 NE 1 (foreign high school diploma) 
and Cla-c NE 1, go to D1. 

Else, go to D1. 



For Participants 



C2. 


What was the main reason you took basic skills or high school completion classes? 




[probe: read likely answer(s)] 






BSREASON 


TO IMPROVE, ADVANCE, OR KEEP UP TO DATE 








ON CURRENT JOB 








TO TRAIN FOR A NEW JOB OR A NEW CAREER 
TO IMPROVE YOUR BASIC READING, WRITING, 


2 






OR MATH SKILLS 








TO MEET A REQUIREMENT FOR A DIPLOMA, DEGREE, OR 






CERTIFICATE OF COMPLETION 


4 




BSREA0S1 


SPECIFY 








A PERSONAL, FAMILY, OR SOCIAL REASON . . 


5 






SOME OTHER MAIN REASON 


91 




BSREA0S2 


SPECIFY 








REFUSED 


-7 






DON'T KNOW 






C5. 


In the oast 12 months, how manv weeks did 
completion classes? 


you attend basic skills or high school 


BSWHEN 


NUMBER 






BSWHENUN 


Unit 








DAYS 


. . . , 1 


(GO TO C6) 
(GO TO C6) 
(GO TO C6) 




WEEKS 


.... 2 




MONTHS 






SEMESTER 


. . . . . 4 


(GO TO C6) 

(GO TO C6) 

(GO TO C50V) 




QUARTER 


. . . . . 5 




OTHER 


... 91 


BSWHENOS 


SPECIFY 






REFUSED 


-7 


(GO TO C6) 
(GO TO C6) 




DON'T KNOW 




C50V. 


How many weeks was that? 






BSWKS 


NUMBER 







Collect number; autocode unit. 
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C6. 



For about how many hours ( per dav/per week) did you attend during the time you were 
going to the classes? 



BSHRS NUMBER □□ 

BSHRSUNT Unit 

PER DAY 1 

PER WEEK 2 

REFUSED -7 

DON'T KNOW -8 



C7. In the past 1 2 months , about how much of your own money would you estimate you paid 

for tuition, books, transportation, child care, and other expenses to take basic skills or high 
school completion classes? 



BSTUITON AMOUNT $□□,□□□ 

REFUSED -7 

DON'T KNOW -8 



If A6 NE 1 (not worked in the past 12 months), 
then go to D1. 

If A7 = 1 and A8 NE 1 (self-employed only), 
then go to D1. 



Cl 3. (Not counting your self-employment,) Was your employer aware that you were taking or 

took the basic skills or high school completion class? 



BSAWARE YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



If Cl 3 NE 1 (employer was not aware), 
then go to D1. 



C14. 



Did your employer . . . 



BSEMPREQ a. 
BSEMPWP b. 
BSEMPSPA c. 
BSEMPPAY d. 



Require you to take these classes? 

Give you time off from work with or without pay? 

Provide classroom space? 

Pay all or part of the cost, including tuition, books, 
and supplies, and other costs like transportation? 



YES 


NO 


R 


DK 


. 1 


2 


-7 


-8 


. 1 


2 


-7 


-8 


. 1 


2 


-7 


-8 


. 1 


2 


-7 


-8 
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D. Credential 



(Not including the classes you told us about earlier,) During the past 12 months , did you 
take any courses that are part of a program, or a series of courses associated with a 
program leading toward... 



D1. 



NO R DK 
2 -7 -8 

2 -7 -8 

1 IfDlaorb = 1 , then ask D2. Else, go toEI. 

For Participants 



CRDEGREE a. 
CRVOCDIP b. 



A college or university degree, such as an 
associate's, bachelor's, or graduate degree? 

A diploma or certificate from a vocational or 
technical school after high school or a formal 
vocational training program 



YES 

. . 1 



1 



D2. In how many of these degree, diploma, or certificate programs were you enrolled during 

the oast 1 2 months ? 



CRPRGNUM NUMBER □□ 

REFUSED -7 

DON'T KNOW -8 



D3. In what type of degree, diploma, or certificate program(s) were you working? 

[CODE UP TO 5 - CATEGORIES CAN BE ENTERED MORE THAN ONCE FOR MULTIPLE PROGRAMS OF THE 
SAME PROGRAM TYPE.] 



CRDIPLOM VOC/TECH DIPLOMA AFTER HIGH SCHOOL, 

BUT BELOW BACHELOR'S DEGREE 1 

ASSOCIATE'S DEGREE 2 

BACHELOR'S DEGREE 3 

MASTER'S DEGREE 4 

DOCTORATE (PHD, EDD) 5 

PROFESSIONAL DEGREE BEYOND BACHELOR'S DEGREE 

(medicine/md; dentistry/dds; law/jd/llb; etc.) . . 6 

ANOTHER DEGREE 91 

CRDIPLOS SPECIFY 

REFUSED -7 

DON'T KNOW -8 
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Now, let's talk about your (credential). 



D5. What was the main reason you were working on the (credential)? 

[PROBE: READ LIKELY ANSWER(S)] 



CRREASON TO IMPROVE, ADVANCE, OR KEEP UP TO DATE 

ON CURRENT JOB 1 

TO TRAIN FOR A NEW JOB OR A NEW CAREER 2 

TO IMPROVE YOUR BASIC READING, WRITING, 

OR MATH SKILLS 3 

TO MEET A REQUIREMENT FOR A DIPLOMA, DEGREE, OR 

CERTIFICATE OF COMPLETION 4 

A PERSONAL, FAMILY, OR SOCIAL REASON, 5 

SOME OTHER MAIN REASON 91 

CRREAOS2 SPECIFY 

REFUSED -7 

DON'T KNOW -8 



If A6 NE 1 (not worked in the past 12 months), 
then go to E 1. 

If A7 = 1 and A8 NE 1 (self-employed only), 
then go to El. 



D1 7. (Not counting your self employment,) Was your employer aware that you were taking or 

took the (CREDENTIAL)? 



CRAWARE yes 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



If D1 7 NE 1, then go to El. 



D18. Did your employer . . . 



» 



CREMPREQ 


a. 


Require you to take a (credential)? 


YES 
. 1 


NO 

2 


R DK 
-7 -8 


CREMPWP 


b. 


Give you time off from work with or without pay? 


. 1 


2 


-7 -8 


CREMPSPA 


c. 


Provide classroom space? 


. 1 


2 


-7 -8 


CREMPPAY 


d. 


Pay all or part of the cost, including tuition, books, 
and other costs like transportation? 


. 1 


2 


-7 -8 



» 
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E. Apprenticeship 



El . During the past 12 months, were you in a formal apprenticeship program leading to 

journeyman status in a skilled trade or craft? 



APPRENTI yes 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



F. Career or Job Related Activities 



FI • Now, I'd like to ask about courses related to a job or career, whether or not you had a job 

when you took the courses. (Please don't include courses you already told me about.) 
Some examples are courses taken at your job, courses taken somewhere else that relate 
to your job or a new career, or courses for a license or certification you need for your job. 
Have you taken any of these in the past 12 months ? 



WRACTY yes 

NO 

REFUSED . . . 
DON'T KNOW 



1 (GO TO F2) 

2 (GO TO FI 9) 

-7 (GO TO FI 9) 

-8 (GO TO FI 9) 



For Participants 



F2. (Not counting courses you took for a credential program,) how many career or job related 

courses did you take during the past 12 months ? 



WRNUM number □□ 

REFUSED -7 

DON'T KNOW -8 



WRCOURSE. Now, I'm going to ask about the names and general subject matter of the course(s) you 
took. By general subject matter we mean the broad topic area, such as business 
management, computer software, auto mechanics, and so on. 



F3. 



What was the course name and what was the general subject matter for (this/each) 
course? 



WRNAME 

WRSUBJ 



NAME SUBJECT 

NAME SUBJECT 

NAME SUBJECT 

NAME SUBJECT 

REFUSED -7 

DON'T KNOW -8 



For a Selected Course 



» 



» 



» 



» 



» 



» 



F4. (Rather than asking you to tell us about aH these courses, the computer has selected one 

course automatically.) Let's talk about (course name). What was the main reason you 
took part in (course name)? 

[PROBE: READ LIKELY ANSWER(S)] 



WRREASON TO IMPROVE, ADVANCE, OR KEEP UP TO DATE 

ON CURRENT JOB 1 

TO TRAIN FOR A NEW JOB OR A NEW CAREER 2 

TO IMPROVE YOUR BASIC READING, WRITING, 

OR MATH SKILLS 3 

TO MEET A REQUIREMENT FOR A DIPLOMA, DEGREE, OR 

CERTIFICATE OF COMPLETION 4 

A PERSONAL, FAMILY, OR SOCIAL REASON 5 

SOME OTHER MAIN REASON 91 

WRREAOS2 SPECIFY 

REFUSED -7 

DON'T KNOW -8 



F6. In the past 12 months, how many weeks did you attend (course name)? 



WRWHEN NUMBER 



□ □ 



WRWHENUN 



WRWHENOS 



Unit 

DAYS 

WEEKS 

MONTHS 
SEMESTER . . . 
QUARTER . . . 

OTHER 

SPECIFY 

REFUSED 
DON'T KNOW . 



1 


(GO 


TO 


F7) 


2 


(GO 


TO 


F7) 


3 


(GO 


TO 


F7) 


4 


(GO 


TO 


F7) 


5 


(GO 


TO 


F7) 


91 


(GO 


TO 


F60V) 


-7 


(GO 


TO 


F7) 


-8 


(GO 


TO 


F7) 



F60V. How many weeks was that? 

WRWKS weeks □□ 



Collect Number; autocode unit. 



F7. For about how many hours (per day/per week) did you attend? 



WRHRS NUMBER □□ 

WRHRSUNT Unit 

PER DAY 1 

PER WEEK 2 

REFUSED -7 

DON'T KNOW -8 
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F8. In the past 1 2 months , about how much of your own money would you estimate you paid 

for tuition, books, transportation, child care, and other expenses to take (course name)? 



WRTUITON AMOUNT $□□,□□□ 

REFUSED -7 

DON'T KNOW -8 



If A6 NE 1 (not worked in the past 12 months), 
then go to FI 8. 

If A7 = 1 and A8 NE 1 (self-employed only), 
then go to FI 8. 



FI 2. ((Not counting your self-employment, )/( Not counting your work-study or assistantship,)) 

Was your employer aware that you were taking or took this course? 



WRAWARE yes 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



If FI 2 NE 1, go to FI 8. 



FI 3. Did your employer . . . 



YES NO R DK 

WREMPREQ a. Require you to take (course name)? 1 2 -7 -8 

WREMPWP b. Give you time off from work with or without pay? .1 2-7-8 

WREMPSPA c. Provide classroom space? 1 2 -7 -8 

WREMPPAY d. Pay all or part of the cost, including tuition, books, 

and other costs like transportation? 1 2 -7 -8 



Go to G1. 



For Non-Participants 



FI 8. In the past 1 2 months , did you have an interest in taking any career or job related courses? 



WRINTRST YES 

NO 

REFUSED . . . 
DON’T KNOW 



1 (GO TO FI 9) 

2 (GO TO BOX BEFORE F25) 
-7 (GO TO BOX BEFORE F25) 
-8 (GO TO BOX BEFORE F25) 



FI 9. Would you say you were very interested, somewhat interested, or slightly interested in 

taking career or job related courses? 



WRHOWINT VERY INTERESTED 1 

SOMEWHAT INTERESTED 2 

SLIGHTLY INTERESTED 3 

REFUSED -7 

DON’T KNOW -8 
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F20. 



WRKNOW 



F21 . 



WRPRTIME 

WRPRCOST 

WRPRCHIL 

WRPRTRAN 

WRPROTH 

WRPROTOS 



F22. 



WRPRGEN 



Of the career or job related courses that you were interested in, did you know of any 
courses you could have taken in the past 1 2 months? 



YES 1 (GO TO F21 ) 

NO 2 (GO TO BOX BEFORE F25) 

REFUSED -7 (GO TO BOX BEFORE F25) 

DON'T KNOW -8 (GO TO BOX BEFORE F25) 



If NUMKID 12 = 0, do not display F21c. 



Now, I'm going to read a short list of things that may have prevented you from taking 
career or job related courses. For each one, please tell me if it was a major obstacle, a 
minor obstacle, or not an obstacle. How about... 

[probe: Was that a major obstacle, minor obstacle, or not an obstacle?] 

[IF RESPONDENTS ANSWER "NO" IN F21e, ENTER "3" AND GO TO BOX BEFORE F25. 

else, probe: Was that a major or minor obstacle? then, collect verbatim response.] 







MA 


Ml 


NOT R DK 


a. 


Time 


1 


2 


3 -7 -8 


b. 


Money or cost 


1 


2 


3 -7 -8 


c. 


Child care 


1 


2 


3 -7 -8 


d. 


Transportation 


1 


2 


3 -7 -8 


e. 


Was there any other obstacle? 


1 


2 


3 -7 -8 



What was that? 



If only one major in F2 1, 
autocode F22 = F21 major. 

Else, display categories rated major in F21 . 

If only one minor and no major in F2 1, 
autocode F22 = F21 minor. 

Else, display categories rated minor in F21 . 

If no major and no minor in F2 1, go to box before F25. 



[DISPLAY RESPONSES IN F21] 

Of the reasons you said were (major/minor) obstacles, what was the main thing that 
prevented you from taking career or job related courses? 



TIME 1 (GO TO F23A) 

MONEY OR COST 2 (GO TO F23B) 

CHILD CARE 3 (GO TO F23C) 

TRANSPORTATION 4 (GO TO F23D) 

(RESPONSE IN F21e) 5 (GO TO BOX BEFORE F25) 

REFUSED -7 (GO TO BOX BEFORE F25) 

DON'T KNOW -8 (GO TO BOX BEFORE F25) 



If A6 NE 1 (not worked in the past 12 months), 
do not display F23Ac and F23Ad. 

If NUMKID 12 = O, do not display F23Bc. 

If F23Ag, F23Be, F23Cc, F23Dd = 1 or 2, 
collect verbatim responses. 
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BEST COPY AVAILABLE 



F23. 



WRTIFAM 

WRTICHOR 

WRTICLHR 

WRTIWORK 



WRTIACTI 

WRTITRAV 

WRTIOTH 

WRSPOTOS 

WRMOTUIT 

WRMOBOOK 

WRMOCHIL 

WRMOTRAN 

WRMOOTH 

WRSPOTOS 

WRCHCOST 

WRCHAVAL 

WRCHOTH 

WRSPOTOS 

WRTRCOST 

WRTRAVAL 

WRTRTIME 

WRTROTH 

WRSPOTOS 



i 



[DISPLAY LIST ASSOCIATED WITH MAIN OBSTACLE IN F22] 

Now, I'm going to read a short list of (time/money or cost/child care/transportation) related 
problems that may have prevented you from taking career or job related courses. For each 
statement, please tell me if it was a major obstacle, a minor obstacle, or not an obstacle 
for you. How about... 

[probe: Was that a major obstacle, minor obstacle, or not an obstacle?] 

[IF RESPONDENTS ANSWER "NO" IN F23Ag, F23Be, F23CC, F23Dd, ENTER "3" AND GO TO BOX 
BEFORE F25. 

else, probe: Was that a major or minor obstacle? then, collect verbatim response.] 

MA Ml NOT R DK 



A. TIME 





a. A desire to spend time with your family. . . . 

b. A need to take care of family duties or 


. 1 


2 


00 

1 

r- 

i 

CO 




chores around the house 

c. Being unable to take courses offered 


. 1 


2 


3 -7 -8 




only during work hours 

d. Work responsibilities that do not permit 
you to take courses either during or 


. 1 


2 


3 -7 -8 




after work hours 

e. Activities outside of work that conflict 


. 1 


2 


3 -7 -8 




with course schedule 


. 1 


2 


3 -7 -8 




f. The travel time to and from courses 


. 1 


2 


3 -7 -8 


B. 


g. Another time related problem 

What was that? 

MONEY OR COST 


. 1 


2 


3 -7 -8 




a. The amount of tuition and fees for courses 


. 1 


2 


3 -7 -8 




b. The cost of books and supplies for courses . . 


. 1 


2 


3 -7 -8 




c. The cost of child care 




2 


3 -7 -8 




d. The cost of transportation 


. 1 


2 


3 -7 -8 


C. 


e. Another money or cost related problem .... 
What was that? 

CHILD CARE 


. 1 


2 


3 -7 -8 




a. The cost of child care 


. 1 


2 


3 -7 -8 




b. The availability of child care 


. 1 


2 


3 -7 -8 


D. 


c. Another problem with child care 

What was that? 

TRANSPORTATION 


. 1 


2 


3 -7 -8 




a. The cost of transportation 


. 1 


2 


3 -7 -8 




b. The availability of transportation 


. 1 


2 


3 -7 -8 




c. The travel time to and from courses 


. 1 


2 


3 -7 -8 




d. Another problem with transportation 

What was that? 


. 1 


2 


3 -7 -8 



If only one major in F23, autocode F24 = F23 major. 
Else, display only categories rated major in F23. 

If only one minor and no major in F23, 
autocode F24 = F23 minor. 

Else, display only categories rated minor in F23. 

If no major and no minor in F23, 
go to box before F25. 
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F24. 



[DISPLAY CATEGORIES IN F23) 

Among the (time/money or cost/transportation/child care) related problems you indicated 
as (major/minor) obstacles, what was the most important obstacle? 

WRPRSPEC [IF NECESSARY, USE DISPLAY AS PROBES] 



A. TIME 

A DESIRE TO SPEND TIME WITH YOUR FAMILY 1 

A NEED TO TAKE CARE OF FAMILY DUTIES 

OR CHORES AROUND THE HOUSE 2 

BEING UNABLE TO TAKE COURSES OFFERED 

ONLY DURING WORK HOURS 3 

WORK RESPONSIBILITIES THAT DO NOT 

PERMIT YOU TO TAKE COURSES EITHER DURING 

OR AFTER WORK HOURS 4 

ACTIVITIES OUTSIDE OF WORK THAT 

CONFLICT WITH COURSE SCHEDULE 5 

THE TRAVEL TIME TO AND FROM COURSES 6 

(VERBATIM RESPONSE IN F23Ag) 7 

B. MONEY OR COST 

THE AMOUNT OF TUITION AND FEES FOR COURSES .... 1 

THE COST OF BOOKS AND SUPPLIES FOR COURSES 2 

THE COST OF CHILD CARE 3 

THE COST OF TRANSPORTATION 4 

(VERBATIM RESPONSE IN F23Be) 5 

C. CHILD CARE 

THE COST OF CHILD CARE 1 

THE AVAILABILITY OF CHILD CARE 2 

(VERBATIM RESPONSE IN F23Cc) 3 

D. TRANSPORTATION 

THE COST OF TRANSPORTATION 1 

THE AVAILABILITY OF TRANSPORTATION 2 

THE TRAVEL TIME TO AND FROM COURSES 3 

(VERBATIM RESPONSE IN F23Dd) 4 



If A6 = 1 (worked in the past 12 months), ask F25. 
If A7 NE 1 or A8 = 1 (not self-employed only), 
then ask F25. Else, go to G1 . 



F25. Did your employer offer any career or job related courses? 



WREMPOFF YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 
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G. Other Formal Structured Activities 



G1. 



SAACTY 



Now, I am going to ask about any other courses where there was an instructor. (Please 
don't repeat any courses (and programs) you have already told us about.) These might 
include things like arts and crafts, sports or recreation, first aid or childbirth, Bible study, 
or any other types of courses we haven't talked about yet. Did you take any of these or 
other courses in the past 12 months? 



yes 1 (go to G2) 

NO 2 (GO TO BOX BEFORE HI) 

REFUSED -7 (GO TO BOX BEFORE HI) 

DON'T KNOW -8 (GO TO BOX BEFORE HI) 



For Participants 

G2. Altogether, how many of these courses did you take during the past 12 months ? 



SANUM NUMBER □□ 

REFUSED -7 

DON'T KNOW -8 



SACOURSE. Now, I'm going to ask about the names and general subject matter of the course(s) you 
took. By general subject matter we mean the broad topic area, such as health, arts and 
crafts, sports, and so on. 

G3. What was the course name and what was the general subject matter for (this/each) 

course? 



SANAME 

SASUBJ 



NAME SUBJECT 

NAME SUBJECT 

NAME SUBJECT 

NAME SUBJECT 

REFUSED -7 

DON'T KNOW -8 



H. Computer-only or Interactive Video-only Instruction on the Job 



If A6 NE 1 (not worked in the past 12 months), 
then go to / 14. 



HI . At your job during the past 12 months , did you learn to do any specific tasks for your job 

using computer-based or interactive video instruction, where you worked only with a 
computer or interactive video without an instructor present? 



CVONLY COMPUTER INSTRUCTION 1 (GO TO H2) 

INTERACTIVE VIDEO INSTRUCTION 2 (GO TO H2) 

BOTH 3 (GO TO H2) 

NONE 4 (GO TO 114) 

REFUSED -7 (GO TO 114) 

DON'T KNOW -8 (GO TO 114) 



For Participants 



Altogether, how many of these computer or video activities did you do during the past 12 
months? 



H2. 

» 

CVNUM NUMBER □□ 

REFUSED -7 

DON'T KNOW -8 



l. Remaining Background 

114. Does your occupation have legal or professional requirements for continuing training or 

education? 



REQUIRMN YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



If A 6 NE 1, autocode 124 = 0 and go to CLOSE. 



124. How many months have you worked for pay or income in the past 1 2 months? 



IBWORKMO MONTHS □□ 

REFUSED -7 

DON'T KNOW -8 



If 124 = 12, then go to box before 128. 



» 



» 



» 



125. At any time during the past 12 months, have you been unemployed and looking for work 

for as long as a month? 



UNEMLOOK YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



Ask 128 except if A7 = 1 and 
A8 NE 1 (self-employed only) and go to CLOSE. 
Else, ask 129 if A 7 = 1 and A8 NE 1 
(self-employed only). 



128. 


Are 


you eligible for the following benefits through any of your current job? 

YES NO R DK 


MEDICAL 


a. 


Medical or hospital insurance? 


i 


2 -7 -8 


SICKPAY 


b. 


Sick leave with full pay? 


i 


2 -7 -8 


VACATPAY 


c. 


Vacation with full pay? 


i 


2 -7 -8 


RETIRMNT 


d. 


Pension plan or retirement program? 


i 


2 -7 -8 



• ERIC 



-47- 



53 



129. 



Do you have the following benefits as a part of your business of being self-employed? 



MEDICAL a. 

RETIRMNT b. 



YES NO R 



Medical or hospital insurance? 1 2 -7 

Pension plan or retirement program? 1 2 -7 



DK 

-8 

-8 



CLOSE. 



Those are all the questions I have. Thank you again for your time. 
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